fix(libSceSystemService): park guest threads during LoadExec's shutdown as exit does - #6237

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

This also fixes the Peace Walker launcher regression reported on #6000 (PPSA18216, same '15ProcessShutdown' terminate at the LoadExec hand-over). That one comes from the audio side: since #6000 the AudioOut2 exports park through LibcAwaitExit_nid_postfix once the shutdown token is set, and the guest's mixer thread pushes every few milliseconds, so with LoadExec's shutdown running without exitRequested the next push throws ProcessShutdown into the title.

Checked on Ubuntu 26.04 (WSL2), GCC 15.2.0, Release, SDL_AUDIODRIVER=dummy, with this PR merged onto main 489466d3f (clean merge). A test process starts an AudioOut2 mixer thread (set the port data, blocking push, repeat), registers a 100 ms shutdown callback in place of the AGC and VideoOut shutdowns, then calls sceSystemServiceLoadExec("/app0/picker/picker.bin", {"--secondary"}); the relaunched image writes a marker.

guest_system_exit, guest_system_relaunch and the four audio_out*_shutdown tests pass on both builds, so none of the existing tests covers a guest audio thread during LoadExec. The test is below if you want it as a regression test (Linux only as written; the 100 ms callback makes it deterministic):

#include "SceTypes.hpp"
#include "prx/libc/include/general/VabiMacros.hpp"
#include "prx/libc/include/Shutdown.hpp"

#include <atomic>
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <filesystem>
#include <fstream>
#include <iterator>
#include <string>
#include <thread>
#include <vector>
#include <sys/wait.h>
#include <unistd.h>

extern "C" {
int APS5_VABI sceSystemServiceLoadExec(const char*, const char* const*);
int APS5_VABI sceAudioOut2ContextResetParam(AudioOut2ContextParam*);
int APS5_VABI sceAudioOut2ContextCreate(const AudioOut2ContextParam*, void*, std::size_t, AudioOut2ContextHandle*);
int APS5_VABI sceAudioOut2ContextPush(AudioOut2ContextHandle, std::uint32_t);
int APS5_VABI sceAudioOut2PortCreate(AudioOut2ContextHandle, const AudioOut2PortParam*, AudioOut2PortHandle*);
int APS5_VABI sceAudioOut2PortSetAttributes(AudioOut2PortHandle, const AudioOut2Attribute*, std::uint32_t);
}

namespace {

constexpr char Marker[] = "ANYPS5_TEST_RELAUNCH_MARKER";
std::atomic<int> pushes{0};
std::vector<float> pcm(512, 0.25f);

void Fail(const char* message) {
    std::fprintf(stderr, "%s\n", message);
    std::fflush(stderr);
    std::_Exit(1);
}

std::filesystem::path OwnExecutable() { return std::filesystem::read_symlink("/proc/self/exe"); }

void AppendMarker(const std::string& line) {
    const char* marker = std::getenv(Marker);
    if (marker == nullptr) Fail("no marker");
    std::ofstream(marker, std::ios::app) << line << '\n';
}

void SlowDriverShutdown() { std::this_thread::sleep_for(std::chrono::milliseconds(100)); }

void StartMixer() {
    AudioOut2ContextParam params{};
    sceAudioOut2ContextResetParam(&params);
    params.num_grains = 256;
    AudioOut2ContextHandle context = 0;
    if (sceAudioOut2ContextCreate(&params, nullptr, 0, &context) != 0) Fail("context");
    AudioOut2PortParam portParams{};
    portParams.data_format = 2u << 8;
    portParams.sampling_freq = 48000;
    AudioOut2PortHandle port = 0;
    if (sceAudioOut2PortCreate(context, &portParams, &port) != 0) Fail("port");
    std::thread([context, port] {
        for (;;) {
            const void* pointer = pcm.data();
            const AudioOut2Attribute attribute{0, 0, &pointer, sizeof(pointer)};
            sceAudioOut2PortSetAttributes(port, &attribute, 1);
            sceAudioOut2ContextPush(context, 1);
            pushes++;
        }
    }).detach();
}

int Drive() {
    const auto self = OwnExecutable();
    const auto secondary = self.parent_path() / ("picker.picker.bin" + self.extension().string());
    std::filesystem::copy_file(self, secondary, std::filesystem::copy_options::overwrite_existing);
    const auto marker = std::filesystem::temp_directory_path() / ("anyps5-audio-loadexec-" + std::to_string(getpid()) + ".txt");
    std::filesystem::remove(marker);
    setenv(Marker, marker.c_str(), 1);
    const pid_t child = fork();
    if (child == 0) {
        execl(self.c_str(), self.c_str(), "--launch", static_cast<char*>(nullptr));
        std::_Exit(127);
    }
    int status = 0;
    waitpid(child, &status, 0);
    std::string text;
    for (int attempt = 0; attempt < 100 && text.find("relaunched") == std::string::npos; ++attempt) {
        std::this_thread::sleep_for(std::chrono::milliseconds(50));
        std::ifstream file(marker);
        text.assign(std::istreambuf_iterator<char>(file), std::istreambuf_iterator<char>());
    }
    std::filesystem::remove(marker);
    std::filesystem::remove(secondary);
    std::printf("launcher: %s %d; marker: %s", WIFEXITED(status) ? "exit" : "signal", WIFEXITED(status) ? WEXITSTATUS(status) : WTERMSIG(status), text.empty() ? "(empty)\n" : text.c_str());
    if (text != "relaunched\n") Fail("the relaunched executable did not start");
    std::puts("Audio LoadExec completed");
    return 0;
}

}

int main(int argc, char** argv) {
    setenv("SDL_AUDIODRIVER", "dummy", 1);
    if (argc == 2 && std::strcmp(argv[1], "--launch") == 0) {
        StartMixer();
        while (pushes.load() < 3) std::this_thread::sleep_for(std::chrono::milliseconds(1));
        LibcRegisterShutdown_nid_postfix(SlowDriverShutdown);
        const char* const arguments[] = {"--secondary", nullptr};
        sceSystemServiceLoadExec("/app0/picker/picker.bin", arguments);
        Fail("LoadExec returned");
    }
    const int first = argc > 0 && std::strncmp(argv[0], "--", 2) == 0 ? 0 : 1;
    if (argc > first && std::strcmp(argv[first], "--secondary") == 0) {
        AppendMarker("relaunched");
        return 0;
    }
    return Drive();
}

add_test_executable(audio_loadexec_tests ${LIBS_INCLUDE_DIR}/tests/AudioLoadExec.cpp)
target_include_directories(audio_loadexec_tests PRIVATE ${LIBS_INCLUDE_DIR})
target_link_libraries(audio_loadexec_tests PRIVATE libSceSystemService libSceAudioOut libkernel libc)

Not checked: Windows (CreateProcessW path), a title run.

AI-assisted: yes (Claude Code).

Thanks for checking the audio path, and for tying it to Peace Walker. I added your run to the description.

AI-assisted: yes

Cross-validation on Windows 11 Pro 26220, NVIDIA RTX 4070 (596.21), MinGW GCC 15.2, PR head a03e023 merged on main 8dc92ae:

PPSA85300 (VLC 0.3.1). Relinked with --windows-fixed-base 0x400000, app0/eboot.bin staged, 120 s runs. On a first launch the restart is reached through APS5_PAD_INPUT_FILE: Cross, Right (Classic look), Options in the setup. setup_finish then calls plat_restart -> sceSystemServiceLoadExec("/app0/eboot.bin", NULL), so on Windows this is the null-list CreateProcessW path. The restart comes 17-24 s in.

• Before (main 8856f18, whose LoadExec and libc shutdown files are identical to 8dc92ae; 1 run): the last line in vlc-ps5.log is VLC-PS5: restarting to see new drives, then on the console:

  [libc] unhandled exception '15ProcessShutdown' thrown from 00007ff86b386ee7
  terminate called after throwing an instance of 'ProcessShutdown'
  

  app.exe exits with 0xC0000409 about 26 s in. No second process starts, and 1163 frames were presented.
• After (2 runs): the old app.exe exits with 0x00000000 at 18-21 s. The new app.exe starts at 20.9 s and 18.9 s and prints its own Transferring control to ELF entry point on the console. In vlc-ps5.log it reaches start: first frame on screen, VLC-PS5: splash hidden (0) and [shader-disk-cache] (10 s): 6 hits, 0 misses, then presents until the limit: 11753 and 12060 frames at about 120 FPS. The old process presented 1361 and 1238 frames. No ProcessShutdown line. Between the two images, 4 [gpu] storage image ... destroyed with GPU results pending lines.
• With fix(libSceSystemService): give LoadExec's next executable the host standard error #5699 (7a4688c) on top: the same in 2 more runs (12071 and 12122 frames in the new process).

Tests. The 7 guest_system_exit tests pass 3/3 each, and loadexec_arguments passes 3/3. Negative control: with main's Export.cpp, LoadExec.cpp, Process.cpp and Shutdown.hpp and the new test, guest_system_relaunch fails 3/3 (Guest thread resumed during exit, then abort()). The non-GPU suite on main + this PR + #5699 (-j8 --timeout 120 -E '^agc_driver_|^video_out_flip') passes 830/830. The host was shared with other jobs, but nothing timed out in this run. tools/check_conventions.py --base origin/main passes and git diff --check is clean.

A Windows note, not caused by this PR. Each failing negative-control run left a suspended picker.picker.bin.exe (one thread, suspended). Replace creates the child with CREATE_SUSPENDED before the shutdown. When the old process ends inside the shutdown, neither ResumeThread nor the catch's TerminateProcess runs. The orphan keeps the copied executable open, and the next guest_system_relaunch then fails in the driver's copy_file until the orphan is killed. This PR removes the cause we hit. The same orphan can still come from a LoadExec with an argument list while another thread is already in exit: LibcStartExit_nid_postfix then parks the LoadExec thread, and the exit ends the process with the child still suspended. That is an edge case for a follow-up, for example a kill-on-close job object that is cleared before ResumeThread. The changed test covers only the argument-list path. VLC's null-list path is checked only by the title run above.

No other concerns.

Written with Claude Code assistance.

Thanks for the Windows run and the negative control. I added it to the description, along with the suspended-child case as a follow-up.

AI-assisted: yes

Cross-validation on Windows 11 Pro build 26300, Intel i7-11700F, NVIDIA GeForce RTX 3070, MinGW GCC, PR head a03e023 merged onto main 82ddae7.

PPSA99360 (PS5CEMU-HAR 4.0.1) is another launcher that goes through LoadExec. After the pad sequence has let its self-update install into /app0, it logs [boxart] stopped for the game and restarts itself. Relinked with --trap-sdk-syscall-stubs, 150 s runs with a fixed controller sequence (Cross, Down, Right, Circle, Up and so on every 1.2 s, from 12 s):

• On main 82ddae7, the launcher ends right there: [update] 4.0.1 installed in /app0 at 45.9 s, [boxart] stopped for the game at 48.8 s, then [libc] unhandled exception '15ProcessShutdown' and terminate called after throwing an instance of 'ProcessShutdown', exit code 0xC0000409 at 51 s. The same abort showed up in an earlier sweep on main 332b0b1 (install 75.9 s, abort 78 s).
• With this PR merged: 2 of 2 runs that reach the same point (install 63.1 s and 48.1 s, stopped for the game 4.8 s and 1.2 s later) end with exit code 0 and a new instance of the launcher starts (it logs [update] 4.0.1 is out again from 0.3 s, and the harness stopped it at its time limit). No ProcessShutdown and no terminate in either log.

Other runs of the same title did not reach the restart (the update did not start in three of them, so they say nothing either way) and are not counted. This is only the Windows path for a relaunch with no list, as screamm already covered for VLC; I did not run the macOS or Linux paths.

AI-assisted: yes

Thank you @DavidSamir, a second launcher that relaunches itself is exactly the kind of case this needed, and with the ProcessShutdown abort on main next to it. I added it to the description.

AI-assisted: yes

A re-check on current main 0dc1562, which now has #5904, with MGS4 (PPSA18194) on Linux, RTX 4070 Ti, pad input. Since #5092 the title also needs #6842 to start, and #6814 is merged in all of these builds for the start-up race.

• main + fix(agc): drain library prebuilds before device replacement #6814 + fix(libc): give the scripting heap hooks blocks from libc's own heap #6842: 2 of 2 runs end at 42–43 s, at the launcher's LoadExec into mgs4/mgs4.elf, with terminate called after throwing an instance of 'ProcessShutdown'.
• The same + this PR: 2 of 2 runs reach the 180 s limit with mgs4.elf running.

AI-assisted: yes

Thanks. GitHub shows this as mergeable, but it conflicts in behaviour with #5978 (db3b406), which landed after your last MGS4 run:

On main, StartExit() no longer runs RunShutdown. With this PR merged as it stands, every LoadExec would skip all shutdown callbacks.

Please:

1. Rebase onto current main.
2. At all three call sites, call LibcStartExit and then LibcRunShutdown, the way exit does now.
3. Add a test that shutdown callbacks still run on LoadExec, one that would fail if RunShutdown were skipped.

A fresh MGS4 run on the rebased branch would also help.

Thanks @oneandonlydean, you're right: since #5978, StartExit no longer runs the shutdown callbacks, so this PR as it stands would skip them on every LoadExec. I've rebased onto current main and added LibcRunShutdown after LibcStartExit at all three call sites. I'm running the tests and an MGS4 run on the rebased branch now, and I'll push with the results.

On the test: guest_system_relaunch already expects the exact marker text shutdown / arguments ... / secondary shutdown / relaunched. Both shutdown lines come from callbacks registered before each LoadExec, so it should fail if RunShutdown is skipped. I'll show that red/green with the push.

AI-assisted: yes (Claude Code)

Thanks. That plan sounds right. A red run of guest_system_relaunch with RunShutdown skipped, alongside the green run, will cover the test ask. We will re-review when you push.

Pushed: rebased onto main 864729d (5b072bc), and e71ade0 calls LibcStartExit_nid_postfix and then LibcRunShutdown_nid_postfix at all three call sites.

• Test: the nine exit and relaunch tests pass 3 of 3. With 5b072bc's two files, which skip RunShutdown, guest_system_relaunch fails and the other eight pass. ^guest_|libc|loadexec passes 402 of 402.
• MGS4 (300 s, pad input, both builds with fix(libc): give the scripting heap hooks blocks from libc's own heap #6842, which MGS4 needs): with this PR the launcher's LoadExec reaches mgs4/mgs4.elf and the game runs to the limit. Without it, the run ends at 43 s with [libc] unhandled exception '15ProcessShutdown'.

The description is updated with the same.

AI-assisted: yes (Claude Code)

Peace Walker (PPSA18216): e71ade0 fixes the launcher hand-over

System: Linux, RTX 3080 Ti (NVIDIA 615.71.09)
Build: main c9e6c45 plus this PR (e71ade0), #6014, #6196, #6842 and #7072, plus two local changes that aren't in any PR (AvPlayer accepting VP9/Opus, and a depth-sample case posted on #6196). Relinked with --executable mgspw/mgspw.elf.

• On the failing build, gdb on SIGABRT shows the thread that aborts: LibcAwaitExit_nid_postfix called from sceAudioOutOutput's ProcessShutdown catch. That matches isnisk's audio-path trace.
• With this PR the new image draws the title and the Kant quote and plays the opening movies. Frame dumps across the run match the earlier run that used my own local fix (which set exitRequested before LibcRunShutdown); I dropped it for this PR.
• 1 run, with Cross pressed every 2 s for the first 2 minutes.

AI-assisted; the runs are from my machine.

Thanks @mrfelek, that's a useful second title. Peace Walker goes through the same ExitAndStartNewProject hand-over as MGS4, and the abort from sceAudioOutOutput's ProcessShutdown catch matches the audio-path trace. I added your run to the description.

AI-assisted: yes (Claude Code)